Data transfer systems such as networks and I/O busses always have latency due to the finite time of signal propagation over physical media. Accordingly, receive buffers must include capacity to handle data that is in transit when a full or BUSY signal is generated indicating that the receive buffer is full. As the distance between transmitting nodes and the receive buffer increases the required latency capacity of the buffer must also increase. Prior art flow control systems generally use either a high/low watermark system or a credit based system.
A typical high/low watermark flow control system is depicted in FIGS. 1 and 2. FIG. 1 depicts a receive FIFO and FIG. 2 is simplified state graph of the operation of the FIFO. If the receive FIFO transmits a busy signal at time t0 and the transit time to the transmitting module is t1 then the transmitting module will continue transmitting until t3=t0+t1. Thus, the "Round trip depth" is all the data that was in transit at t0 plus the data that was transmitted between t0 and t3. Additional data will also be transmitted due to logic delays in generating and responding to the BUSY signal.
The prior art flow control protocol uses FIFO high and low watermarks to control the state of a receive link. If the FIFO is above the high watermark the link is busy, and BUSY command symbols are transmitted. If the FIFO is below the low watermark the link is ready, and READY command symbols are transmitted.
At system start-up the receive FIFO is empty and below the low watermark. In this flow control state (FCS), the receiver is ready and the READY command signals are transmitted regularly as required. When receive data arrive, they are pushed into the FIFO, and their availability is reported to the output side. When output is unblocked, the receive data is pulled from the FIFO as soon as available. If the output path is blocked or running at a lower transfer rate, data will accumulate in the FIFO.
When the accumulated data reaches the high watermark, the state of the link changes from inbound READY to inbound BUSY. This state change causes the keep alive protocol to send a BUSY signal to stop the flow of data. If there is insufficient space above the high watermark (less than a round trip delay) or the BUSY command is corrupted, receive data may overflow the FIFO. This flow control relies on the buffer space above the high watermark to prevent FIFO overflow by compensating for the delayed response to the BUSY. When sufficient data has been pulled from the FIFO to bring it below the low watermark, the state will return to inbound ready causing the keep alive protocol to send a READY command symbol to restart the flow of data. The storage below the low watermark is used to prevent data underflow.
The FIFO can be sized to support a desired link length by computing the round trip propagation delay of the link. The four link delay components are:
1. Local Node logic Delay (LND) (300 to 450 ns) PA1 2. Outbound physical media delay (PMD) (5 ns/meter) PA1 3. Remote Node logic Delay (RND) (300 to 450 ns) PA1 4. Inbound physical media delay (PMD) (5 ns/meter)
On short links (less than 30 to 50 meters) the propagation delay is dominated by logic delay in the local and remote nodes. The longer links are dominated by delay in the physical media. To provide full link performance the FIFO must be sized 20 to have a full link round trip delay of characters above the high water-mark and the same amount below the low watermark. The table of FIG. 3 lists a few link lengths and the required FIFO depth. The prior art flow control protocol requires equal amounts of storage above and below the watermarks. For long links, the storage required exceeds the on-chip storage available in current ASIC technology.
Prior credit based flow control protocols issue fixed credits to sending nodes. However, these systems have major start-up and lost-credit problems. At start-up the receive FIFO remains empty until a transmitting node receives a credit and sets up its credit management protocol. If a credit is lost, the transmitting node will wait for a credit and the receiving node will wait for a transmission. The complexity of initialization and error recovery to recover lost credits are major drawbacks of a credit based system.